Problem Statement

Reduce landing page drop-off of Refyne Gold by 50%

Overview

Refyne Gold

Refyne Gold Savings is an online way to buy and save gold in an insured vault. It can be purchased in multiple transactions, of any denomination (minimum of INR 10), and accumulated over time. One can redeem their savings as cash into their bank account when needed. All these gold are 24K and are certified by the government.

User Research

We witnessed a ~93% drop on the Refyne Gold Landing Page.

​

Upon further investigation, we found that approximately 62% of the users initially had difficulty grasping the product's concept, with some mistaking it for commodity market investments. However, after receiving further explanation about what the product entails, these users proceeded to engage in transactions.

Hypothesis

If we introduce a product walkthrough screen before the landing page then our drop-off rates will reduce to ~50% due to better product understanding and increased user awareness.

Goal

The goal of the experiment is to reduce the landing page abandonment rate to 50%, thereby boosting the volume of "Buy Gold" transactions. By achieving this, an increased number of users will successfully reach the Refyne Gold page, allowing them to explore the product with enhanced awareness and comprehension.

Success Metrics

1. Worst Case Scenario: landing page drop-off reduces by 25%
2. Best Case Scenario: landing page drop-off reduces by 50%

Health Metric

1. Number of Buy-Gold transactions

Experiment Design

What are we testing?

User Calling Summary

Funnel Data

External Benchmark

In-app stories for product walkthroughs have resulted in remarkable click-through rates (upwards of 30%) and notable response rates (upwards of 70%). Reference: Here​

Variation Design

Control Version

No change to the existing user flow

Test Version - 1

Introduce a three-card in-app story with the last card re-directing to the Refyne Gold landing page.

Further Iterations:

The idea was to initially test a single version, followed by successive iterations of the copy and design in subsequent versions, to understand the impact of textual content and visual layout on the landing page conversion rates.

Audience & Sample Size

Audience Segment

Incoming traffic that clicks on the Refyne Gold thumbnail

Sample Size

Duration of the test

1st April to 30th April, or till we're statistically significant, whichever is earlier.

Implementation and A/A Test

Platform: VWO

Technical Specifications:

Results: [Pending]

Post Experiment

Experiment Result

1. The drop-offs were reduced from 93% to 37%

Release Decision

Go ahead and scale the experiment and implement the new in-app stories for the product walkthrough before the Refyne Gold Landing Page.

Learnings

1. Despite the significant reduction in the drop-off rates, the number of "Buy Gold" transactions did not increase proportionately.
2. The team is still unsure if the walkthrough was enough to solve the initial level of awareness among the users.
3. ~50% of the users who clicked on the Refyne Gold thumbnail tried to change the language of the app in the process.

Next Steps

1. Conduct more experiments with more copy and design variation - include more trust elements in both
2. Conduct more experiments with vernacular copies for in-app stories and other app elements
3. Conduct surveys and calls to understand the drop-off on the "buy gold" page and the "transaction summary" page.
4. 1. One interesting observation that was uncovered during the initial user calls was that users were not able to understand the GST component on the transaction summary page. They had questions like - Why should I pay for GST while buying Gold? Will I be charged GST while selling Gold as well?

Roll-out of In-app story for Refyne Gold

1. Phase - 1: Experiment completed
2. Phase - 2: Live for all new companies + companies onboarded in the last 3 months
3. Phase - 3: Live for companies onboarded before the last 3 months
4. Phase - 4: Live for companies who have technically integrated their ERPs with Refyne

​